Digital imaging systems are becoming increasingly common both in the field of still photography, as well as in the field of motion imaging as is evidenced by the proliferation of digital cameras and video recorders, as well as services for digitizing photographic film.
Color digital imaging devices generally utilize an array of color sensors to capture the image information. The sensors typically have spectral sensitivities that correspond roughly to the red, green and blue portions of the visible spectrum. Alternatively, sensors with other spectral sensitivities such as cyan, magenta and yellow, or cyan, magenta, yellow and green have also been used. Typically, the exposure values captured by the color sensors are integerized to form quantized color values (code values) and stored in memory for later processing. The later processing steps typically include color correction, as well as a variety of spatial processing steps such as sharpening and/or noise removal.
Sensors used for image capture, including both photographic film and solid state sensors, differ fundamentally from the human visual system in that they lack the ability to adapt to the ambient illumination. While the human visual system adapts to both the luminance and chromatic content of the scene illuminant, image capture systems require a number of compensation mechanisms to mimic such adaptation. These compensation mechanisms can be broadly divided into two classes—pre-exposure mechanisms and post-exposure mechanisms. Pre-exposure correction mechanisms can be thought of as “light altering” in that they change the light captured by the image sensor, whereas post-exposure correction mechanism can be thought of as “signal altering” in that they are used to modify the signal captured by the image sensor. Some degree of pre-exposure compensation to the average illumination level is accomplished using the exposure controls—aperture and shutter speed—available on most cameras. These controls provide a basic level of adjustment over the exposure given to the sensor(s), but they are seldom reliable enough to produce perfectly balanced pictures in every instance. Furthermore, camera exposure controls provide no mechanism for compensation for variations in the chromaticity of the ambient illumination. The only pre-exposure mechanism that attempts to correct for illuminant chromaticity is the use of color correction filters. Again, such corrections are generally only approximate since color correction filters are designed for specific discrete illuminant spectral power distributions.
Pre-exposure compensation mechanisms have historically relied on the skill and sophistication of the photographer for their success, but modern automatic exposure metering systems are able to achieve reasonably good results, even in the hands of a neophyte. Since pre-exposure compensation mechanisms only provide a partial solution to the problem of illuminant variability, most images require further compensation for the scene illuminant variability using post-capture mechanisms. There are numerous techniques available to accomplish such compensation. In the traditional silver-halide-negative photography arena, post-exposure illuminant compensation has been accomplished by adjustment of the enlarger or printer lamphouse filtration and control of the printing exposure. The magnitude of such adjustments can be determined by trial and error, or by algorithmic corrections based on statistical measurements of the negative transmittance. In some cases, the “post-exposure correction” may be specified before the exposure is made, as in the case of specifying white-balance gain corrections by selecting a white-balance setting on a digital camera. However, such corrections should not be confused with “pre-exposure corrections” since the corrections are not actually applied until after the image is captured.
Digital imaging opens up many additional possibilities for post-exposure illuminant compensation. Such compensation is generally performed using mathematical transforms applied to the digital image data. For example, white balance adjustments can be applied in a digital camera, or adjustment to the scanned densities of a color negative can be applied in a digital photofinishing system. These transforms are generally applied to a numerical encoding of the scene colors. Many such encodings—both device dependent and device independent—are available to represent colors.
The intent of balancing transforms in digital imaging applications is usually to modify the image in a manner that is consistent with the adaptation mechanisms in the human visual system. Typically, simple transforms consisting of either a multiplicative scaling factor (gain adjustment) in a linear color encoding or an additive shift in a logarithmic color encoding are used to map a particular visually neutral reference in the scene to a set of aim coordinates in the color encoding. The same transform that is used to map the reference neutral is also applied to all other colors in the image.
Since most scenes do not contain a reference neutral patch that can be used for the balancing operation, a neutral reference is usually estimated by analysis of the scene content. Algorithms that perform this scene analysis to determine an estimate of the necessary correction are sometimes referred to as “scene balance algorithms.” (For example, see “Automatic Color Printing Techniques,” Image Technology, April/May 1969, pp. 39-43; U.S. Pat. No. 4,101,217 assigned to AGFA-Gevaert A. G.; U.S. Pat. Nos. 4,707,119 and 4,984,013 assigned to Fuji Photo Film Co., Ltd.; U.S. Pat. No. 4,945,406 assigned to Eastman Kodak Company; and U.S. Pat. No. 5,016,043 assigned to Gretag Systems.) Scene balance algorithms can vary widely in their complexity, as well as in the accuracy of their results. They typically involve analyzing the distribution of overall exposure levels and relative color signal levels in an image to determine the appropriate level of exposure and color balance compensation that is needed. Frequently, these algorithms work by first computing a low-resolution version of the image and then analyzing that image.
Even if the balancing transform produces a perfectly balanced result for the reference neutral, the results obtained for other image colors may not be consistent with human visual system adaptation, and would therefore be perceived as an error. The extent of these errors is a function of the magnitude of the balancing correction (a function therefore of the scene illuminant) and of the color encoding in which the balancing transform is applied.
For example, consider the case where an image is captured by a digital imaging system under a 18,000K daylight illuminant, which might correspond to a shadow scene that was illuminated with blue skylight and no direct sunlight. If the imaging system were optimized to produce well-balanced imaged from a 5000K daylight source, the unbalanced image would have a significantly blue cast. Consider the case where a color balance correction is applied by transforming the image to a linear RGB color space having the primaries of the well-known sRGB color space, and scaling the resulting RGB values using multiplicative scale factors such that a neutral scene object were perfectly corrected. If the aim of the color balance correction is to produce an image that is identical to one where the scene were captured using the nominal 5000K daylight source, the color errors can be determined by computing the color difference between the aim colors and the balanced image colors. The RMS ΔE*ab color error calculated for a set of about 400 test patches corresponding to representative scene reflectance spectra, was found to be 12.07. FIG. 1 is a CIELAB a*-b* plot showing these color errors for each of the test patches.
If in the example above, the sRGB primaries were replaced by the primaries associated with the RIMM RGB color space described in ANSI/I3A IT10.7466 “Electronic Still Picture Imaging—Reference input medium metric RGB color encoding (RIMM-RGB),” the RMS ΔE*ab color error calculated for the same test patches, was found to be 9.40. FIG. 2 is a CIELAB a*-b* plot showing these color errors for each of the test patches. While this represents an improvement in the RMS color error over those associated with the sRGB primaries, they are still far from perfect.
Spaulding et al. have disclosed a method for applying scene balance corrections in a digital imaging system by transforming to a standard color space for performing the analysis of the digital image (see commonly-assigned U.S. Pat. No. 6,243,133). This method has the advantage that a single scene balance algorithm can be used for many different digital imaging devices without needing to retune the algorithm for each device. However, with the method of Spaulding et al., the actual correction step is applied to the digital image in the input color space. Therefore, the resulting color errors will have the same inherent characteristics as those resulting from applying the scene balance correction algorithm directly in the input color space.